System and method for managing graphics applications

[0001] The present invention relates to computer graphics and more particularly to a system and method for managing graphics applications.

[0002] Multi-monitor display systems have seen an increasing growth in popularity over the past several years. These display systems are advantageous because, relative to a standard monitor, they provide more screen real estate. Accordingly, graphics applications have been created that are able to understand this environment and create graphics for display on monitors fed by multiple graphics pipes, that is, a multi-monitor display system that is driven by more than one graphics pipe.

[0003] Unfortunately, there are a large number of graphics applications that are only capable of interacting with one graphics pipe. Thus, these applications are not able to take full advantage of the multi-monitor display systems driven by multiple graphics pipes. Moreover, graphics data from these applications may be distorted if displayed on a non-planar, multi-monitor display. Additionally, window managers are not capable of concurrently interacting with graphics applications that are unaware of the multi-monitor environment and applications that are aware of the multi-monitor environment.

[0004] From the foregoing, it may be appreciated by those skilled in the art that a need has arisen for a technique to provide multi-monitor capability to graphics applications that are unaware of the multi-monitor environment, especially if they are to be used in a non-planar environment. According to the present invention, a system and method for managing unaware graphics applications is provided that substantially eliminate or greatly reduce disadvantages and problems associated with conventional graphics applications.

[0005] In some embodiments, a system for managing graphics applications includes a graphics server and a view modifier. The graphics server is operable to manage the conveyance of graphics data between an aware graphics application and graphics pipes having different display directions. The view modifier is operable to modify graphics data from an unaware graphics application to account for non-planar display of the graphics data.

[0006] In certain embodiments, a method for managing graphics applications includes receiving graphics data from an unaware graphics application and conveying the graphics data to at least one of a plurality of graphics pipes having different display directions. The method also includes modifying the graphics data to account for non-planar display of graphics data.

[0007] The present invention has a variety of technical features. For example, in some embodiments, a graphics server may manage an application that is unaware of the multiple monitor environment, allowing the application to use the monitors as a single large display, which expands the usefulness of multi-monitor displays. One aspect of this management may, for example, include the ability to homogenize the capabilities of heterogeneous graphics pipes, which allows the graphics data from the unaware application to be displayed in the same manner on each monitor. As another example, in certain embodiments, a graphics server may manage graphics applications that are aware of the multiple monitor environment and graphics applications that are unaware of the multiple monitor environment, which again expands the usefulness of multi-monitor displays. Furthermore, a graphics server may concurrently manage applications that are aware and unaware of the multiple monitor environment, which again expands the usefulness of multi-monitor displays. Moreover, aware applications may be run with unaware applications while imposing little overhead on the aware applications. As an additional example, in some embodiments, coherent window management is provided for unaware and aware graphics applications, which again expands the usefulness of multi-monitor displays. As another example, in certain embodiments, unaware applications may query and manipulate the resources of aware applications, and aware applications may query and manipulate the resources of unaware applications. Hence, an application manager may be determined by the type of resource being manipulated rather than the type of application that is manipulating it. As a further example, in particular embodiments, graphics data from an unaware graphics application may be modified based on the display direction of each graphics pipe. Thus, the graphics data may be displayed more accurately, which again increases the usefulness of multi-monitor displays.

[0008] Of course, some embodiments may possess none, one, some, or all of these technical features and/or additional technical features. Other technical features may be readily apparent to those skilled in the art from the following figures, written description, and claims.

[0021] FIG. 1 illustrates one embodiment of a system 10 for managing graphics applications. In general, system 10 includes clients 20, a graphics server 30, and a display system 40, which includes graphics pipes 50 and monitors 60 in this embodiment. Clients 20 generate graphics data, such as for example geometric primitives, that need to be displayed. Graphics server 30 is responsible for managing the conveyance of the graphics data from clients 20 to display system 40. Graphics pipes 50 of display system 40, in turn, are responsible for converting the graphics data into signals that drive monitors 60, which display the graphics data according to the signals.

[0022] As illustrated, monitors 60 of display system 40 are displaying windows 70, each of which contains a graphic 72. For example, window 70a contains graphic 72a, which is a circle in this illustration. As another example, window 70z contains graphic 72z, which is text in this illustration. For its part, window 70b is split between monitors 60a and 60b. Accordingly, part of window 70b is being displayed by monitor 60a and part of window 70b is being displayed by monitor 60b. Graphics server 30 may facilitate this by conveying part of the graphics data for window 70b to graphics pipe 50a and the other part of the graphics data for window 70b to graphics pipe 50b, by conveying all of the graphics data for window 70b to both of graphics pipes 50a–b, along with instructions on how to clip the graphics data, or by any other appropriate technique. Accordingly, in this embodiment, monitors 60 may be used as a single large display.

[0023] In general, monitors 60 may be used as individual display devices, as a single display device, or as a combination of a multiple monitor display device and individual display devices. Furthermore, a configuration file may be used to specify the arrangement and/or operations of monitors 60, which may have a regular or non-regular shape. Note that for objects displayed on more than one monitor, there may be an overlap of the object on both monitors, a gap between the object on both monitors, or a direct representation on both monitors. Also note that clients 20 may or may not be aware of the number of graphics pipes 50 and, hence, the distribution of graphics data between graphics pipes 50 and/or monitors 60.

[0024] In more detail, each of clients 20 includes a graphics application 22 and an interface library 24. Graphics applications 22 may be any appropriate type of computer software that generates graphics data, such as for example text, objects, pictures, and/or video. Moreover, the graphics data may be two-dimensional (2D) or three-dimensional (3D). Examples of such graphics applications include Netscape™, Origami, and Performer™. As noted previously, graphics applications 22 may be aware or unaware of the number of graphics pipes 50 and/or monitors 60. An example of an unaware application is Netscape™, and an example of an aware application is Performer™. Interface libraries 24 are responsible for receiving graphics information calls—graphics data, initialization data, commands, or any other appropriate graphics related function—from graphics applications 22 and converting them into an appropriate format for graphics server 30. In particular embodiments, interface libraries 24 are X-libraries, although any other appropriate type of library may be used. Interface libraries 24 may be different libraries or different instances of the same library. In particular embodiments, however, one or more of graphics applications 22 may not require an interface library.

[0025] Graphics server 30 includes an unaware application manager 32 and an aware application manager 36. Unaware application manager 32 is, in general, responsible for managing the conveyance of graphics data from clients 20 that are not capable of appreciating the number of graphics pipes 50, this ability determined by the type of graphics application 22 of each client 20. Such applications are typically only be able to interact with one of graphics pipes 50, although some may be able to interact with just a fewer number of graphics pipes than are being used in the system. To accomplish this, unaware application manager 32 aggregates the characteristics of two or more of graphics pipes 50 and presents them to these clients as a consolidated unit. Furthermore, unaware application manager 32 is responsible for conveying the graphics data from these clients to the appropriate graphics pipes 50. Conveying the data may include dividing the data appropriately between the graphics pipes, copying the data between the graphics pipes, or any other appropriate technique. Accordingly, unaware graphics applications may view graphics pipes 50 as being fewer in number than there actually are. In contrast, aware application manager 36 is, in general, responsible for managing the conveyance of graphics data from clients that are capable of appreciating the number of graphics pipes 50. To accomplish this, aware application manager 36 presents the characteristics of each of graphics pipes 50 to these clients and conveys the graphics data from these applications to the appropriate graphics pipes 50. Conveying the data may include distributing the data appropriately between the graphics pipes, copying the data between the graphics pipes, or any other appropriate technique. Unaware application manager 32 and aware application manager 36 may send the graphics data through graphics pipes 50 by multicast, broadcast, direct messaging, or any other appropriate technique.

[0026] In particular embodiments, unaware application manager 32 and aware application manager 36 perform their duties by using respective dispatch tables. Dispatch tables may include a variety of functions that are able to respond to calls from clients 20, such as for example establishing a connection with clients 20, creating a window on monitors 60 for clients 20, and drawing graphics data, such as for example lines, arcs, and rectangles, for clients 20. Some of the functions performed in response to the calls may be carried out by different parts of graphics server 30. In particular embodiments, the dispatch tables know which function to execute based on identifiers included by interface libraries 24 with the graphics information.

[0027] In certain embodiments, the dispatch table for unaware application manager 32 may accomplish its function by determining the division of graphics data between graphics pipes 50 and then accessing functions in the dispatch table for aware application manager 36 for each of graphics pipes 50, substituting per-pipe resources as required. For example, if an unaware application issues a resource creation request, the dispatch table for unaware application manager 32 actually creates a resource for each graphics pipe 50 by calling the dispatch table in application manager 36 once for each graphics pipe. The application will only know about one of the created resources. For later resource requests, the dispatch table looks up the related per pipe resources and substitutes them as required when it calls the other dispatch table for each graphics pipe 50.

[0028] As mentioned previously, unaware application manager 32 and aware application manager 36 are responsible for conveying graphics data from graphics applications 22 to the appropriate graphics pipes 50. Accordingly, if one of graphics applications 22 wants to draw an object on monitor 60a, unaware application manager 32 or aware application manager 36 may convert the graphics data into an appropriate format for graphics pipe 50a and send the formatted data to the graphics pipe. In particular embodiments, graphics server 30 may use X-server technology for managing 2D graphics data and openGL technology for managing 3D graphics data.

[0029] Graphics server 30 may be implemented on any type of computer. In certain embodiments, graphics server 30 is implemented on an Onyx™ server from Silicon Graphics. Furthermore, in particular embodiments, graphics server 30 may include an X-server from Silicon Graphics with Xorg's Xinerama extension. Graphics server 30 may include logical instructions encoded in random access memory, in a field programmable gate array, in a compact disk, or in any other type of volatile or non-volatile electromagnetic or optical information storage media.

[0030] As mentioned previously, display system 40 includes graphics pipes 50 and monitors 60. In general, graphics pipes 50 are responsible for converting the graphics data into signals that drive monitors 60. As illustrated, each of graphics pipes 50 drives one of monitors 60. Graphics pipes 50 may be video adapters, graphics accelerators, and/or any other type of device for converting graphics data into signals that drive one or more of monitors 60. In particular embodiments, each of graphics pipes 50 may have an associated display driver, which is typically a piece of software residing between the graphics pipe and the graphics server/graphics library, to abstract the quirks of a particular graphics pipe so that a common interface may be presented to the graphics server/graphics library for drawing. Furthermore, in certain embodiments, each of graphics pipes 50 may include a framebuffer, which is a component into which the graphics application draws and out of which the monitor reads. The size of the framebuffer may correspond to the number of pixels and the capability sets available to display the pixels for the particular graphics pipe. Monitors 60 receive the signals from graphics pipes 50 and generate illumination detectable by the human eye based on the signals. Monitors 60 may be cathode-ray tube (CRT) displays, liquid crystal displays (LCDs), and/or any other type of device for generating visual information. Moreover, in certain embodiment intervening devices may be placed between monitors 60 and graphics pipes 50.

[0031] In operation, when one of graphics applications 22 wants to have graphics data displayed, it establishes a connection to graphics server 30. To accomplish this, the graphics application calls the appropriate function in the associated interface library 24, which generates an appropriate connection request to graphics server 30. The connection request also includes an indication of whether the application wants to be aware or unaware of the number of graphics pipes 50. Interface library 24 may determine this, for example, by examining a run-time environment variable. When the connection request arrives at graphics server 30, graphics server 30 determines whether the connection request is for a client that wants to be aware of each of graphics pipes 50 or unaware of each of graphics pipes 50. If graphics server 30 determines that the client does not want to be aware of graphics pipes 50, graphics server 30 assigns the connection request to unaware application manager 32, the application manager typically remaining assigned to the connection until the connection closes. If, however, graphics server 30 determines that the connection request is from a client that wants to be aware of graphics pipes 50, the graphics server assigns the connection request to aware application manager 36. The assigned application manager will then generate a connection reply containing the appropriate information regarding graphics pipes 50 for the requesting graphics application.

[0032] When unaware application manager 32 receives a connection request, unaware application manager 32 determines the characteristics of graphics pipes 50 and aggregates them for the requesting client 20. For example, unaware application manager 32 may determine that the resolution of each display area is 640×480. Accordingly, unaware application manager 32 may report that the resolution of the display area is 1920×480. The requesting client 20, therefore, would think that there is one large display area when, in fact, there are three smaller display areas.

[0033] In contrast, when aware application manager 36 receives a connection request, it determines the underlying characteristics of graphics pipes 50 and reports them back individually to the requesting graphics application. For example, if the resolution of each display area is 640×480, the requesting client would learn that there are three 640×480 display areas.

[0034] Once the requesting client 20 has determined the capabilities of display system 40, the graphics application of the client may initiate the creation of a window for displaying graphics data. To accomplish this, the graphics application may make an appropriate call to the associated interface library 24, which generates an appropriate window creation request for graphics server 30. When the window request is received by graphics server 30, the window request is associated with the appropriate one of unaware application manager 32 or aware application manager 36 based on the assignment of the connection request. The appropriate application manager may then create the window for the client and notify the client that the window is available.

[0035] The graphics application of the client may then begin sending graphics data to the graphics server 30. To accomplish this, the graphics application calls the associated interface library 24, which generates an appropriate graphics data request for graphics server 30. When the graphics data is received by graphics server 30, it, as with the connection request and the create window request, is processed by the appropriate application manager, and is conveyed to the appropriate graphics pipes 50. As discussed previously, unaware application manager 32 may divide the graphics data from a client between graphics pipes 50, and, in contrast, aware application manager 36 may distribute the graphics data from a client to the appropriate ones of graphics pipes 50.

[0036] Note that an application capable of being aware of graphics pipes 50 may access graphics server 30 through an unaware connection, making it unaware of graphics pipes 50. Graphics server 30 may manage such a client as an unaware application. Also note that a client accessing graphics server 30 through an unaware connection may still access aspects of graphics server 30 that are normally hidden from unaware applications. In particular embodiments, components of client 20 may make explicit calls to the application manager.

[0037] This embodiment of the present invention has a variety of technical features. For example, a graphics server may manage an unaware graphics application by allowing it to use multiple monitors as a single large display, resulting in a larger framebuffer size. Moreover, the graphics of such an application may be dragged across or overlap graphics pipe boundaries. As another example, a graphics server may manage unaware and aware graphics applications. Accordingly, multi-monitor displays may be readily used with both types of graphics applications, which expands the usefulness of multi-monitor display systems. Moreover, the graphics server may simultaneously manage unaware and multiple applications. Accordingly, these graphics applications may be used in conjunction with each other, which again expands the usefulness of multi-monitor systems. A variety of other technical features exist.

[0038] In particular embodiments, aware applications may be run with unaware graphics applications while imposing little overhead on the aware graphics applications. This allows the latter to function normally even though both type of applications are being managed.

[0039] In certain embodiments, a graphics application may open any number of aware and unaware connections, such as for example an unaware connection for a graphical user interface (GUI) and an aware connection for graphics. This allows different contexts of the application to have access to the multi-monitor display system, if desired.

[0040] In some embodiments, unaware applications may query and manipulate the resources of aware applications and/or aware applications may query and manipulate the resources of unaware applications, which allows these applications to interact with each other. To accomplish this, when unaware application manager 32 receives an aware resource request, it, perhaps after failing when trying to look up the real window identifier mapping, calls aware application manager 36 to handle the request instead. A similar approach may be taken for the case in which aware application manager 36 tries to handle an unaware resource request. Thus, the application manager for the request is chosen based on the type of resource being manipulated rather than based on whether the client making the request is aware or unaware. Such functionality may also be useful if an unaware graphics application accidentally obtains an aware window identifier or an aware graphics application accidentally obtains an unaware window identifier.

[0041] Although system 10 illustrates one embodiment of a system for managing graphics applications, other embodiments may have less, more, and/or a different arrangement of components. For example, in some embodiments, unaware application manager 32 may be part of aware application manager 36. As another example, in certain embodiments, parts of unaware application manager 32 and/or aware application manager may be external to graphics server 30. As a further example, in particular embodiments, one or more of graphics pipes 50 may drive more than one of monitors 60. As another example, in certain embodiments, more than one of graphics pipes 50 may drive one of monitors 60, possibly through the use of an intervening compositor. As an additional example, in some embodiments, display system 40 may use projectors and screens instead of monitors 60. In general, therefore, display system 40 may use any type of devices for generating visual information from graphics data. As another example, in particular embodiments, one or more of clients 20 may be coupled to more than one graphics server without their knowledge, through the use of proxy libraries associated with the client and the graphics server. The proxy libraries allow the connection requests, window creation requests, and graphics data to be conveyed correctly to each of the graphics servers and/or associated graphics pipes while aggregating the responses from the graphics servers and/or graphics pipes. Accordingly, the proxies allow the graphics applications to think it is interacting with fewer graphics pipes than there actually are. A variety of other examples exist.

[0042] FIG. 2 illustrates an embodiment of a system 12 for managing graphics applications. As with system 10, system 12 includes clients 20, graphics server 30, and display system 40, which includes graphics pipes 50 and monitor 60. In this embodiment, however, system 12 also includes a window manager 80, which is responsible for managing windows 70. As can be seen on monitors 60, windows 70 are now bordered by frames 74, which are generated and managed by window manager 80. Frames 74 allow a user to resize and/or move windows 70. For example, frames 74 may provide the commonly known minimize, restore, and/or maximize functions. Additionally, frames 74 may provide a close function for the graphics application.

[0043] In operation, graphics applications 22 generate connection request calls, window request calls, and graphics data calls to the associated interface libraries 24, which generate appropriate graphics server requests for graphics server 30. In this embodiment, however, when one of unaware application manager 32 or aware application manager 36 generates a window reply, it also notifies window manager 80 that a window has been created. The window manager 80 may then begin to manage the window, such as for example putting frames around the window and positioning it on monitors 60. Window manager 80 may use the functions in the appropriate application manager to manage the windows. If window manager 80 detects a command to resize one of windows 70, the window manager will resize the associated frame 74 for the window. Graphics server 30 may instruct the client as to the graphics data that should be sent for the modified window 70. Additionally, if window manager 80 detects that one of frames 74 has been grabbed by a user, window manager 80 is responsible for moving the frame and accompanying window 70 on monitors 60. Note that if one of windows 70 has been moved on top of another one of windows 70, window manager 80 is responsible for stacking the windows, and graphics server 30 may instruct the client generating the graphics data for the overlaid window that it no longer needs to generate the graphics data that cannot currently be observed. Window manager 80 may include the Motif Window Manager, the 4 Dwm by Silicon Graphics, or any other functions for managing windows.

[0044] In the illustrated embodiment, window manager 80 includes a window manager proxy 82. Window manager proxy 82 may actually be part of window manager 80 or may be an interface between graphics server 30 and window manager 80. Window manager proxy 82 is responsible for receiving window-related information from unaware application manager 32 and aware application manager 36 and converting it into an appropriate format for window manager 80. Additionally, window manager proxy 82 is responsible for receiving window management commands from window manager 80 and converting them into an appropriate format for unaware application manager 32 and aware application manager 36. In particular embodiments, window manager proxy 82 knows whether an application is aware or unaware of graphics pipes 50 by the path upon which a request arrives. Thus, window manager proxy 82, in effect, allows window manger 80 to have an aware and an unaware connection to graphics server 30, allowing window manager 80 to concurrently manage windows for unaware and aware graphics applications. In certain embodiments, however, window manager 80 and/or window manager proxy 82 may connect to multiple graphics servers, each having an application manager.

[0045] Accordingly, window manager 80 may include a window manager that does not appreciate that some graphics applications do not understand that there are multiple graphics pipes and that some graphics applications do understand that there are multiple graphics pipes or may be a window manager that does have such an appreciation. Furthermore, window manager 80 may itself be unaware of graphics pipes 50 for unaware applications and aware of graphics pipes 50 for aware applications. In general, window manager proxy 82 may allow any type of window manager to be used.

[0046] This embodiment of the invention provides concurrent and coherent support for window management for unaware and aware graphics applications. Accordingly, windows management is available for unaware and aware graphics applications operating at the same time, and the management for such applications will be consistent since it is performed by one window manager.

[0047] FIG. 3 illustrates an embodiment of a system 14 for managing graphics applications. As with system 10, system 14 includes clients 20, graphics server 30, and display system 40. In this embodiment, however, client 20z includes a display system proxy 26z and a graphics library 28z. Display system proxy 26z is operable to intercept and convey the graphics data from graphics application 22z to graphics pipes 50, and graphics library 28z is operable to convert the graphics data into an appropriate format for graphics pipes 50. The data is sent to these graphics pipes using render threads, and proxy 26z may create one thread per pipe per context per application. Accordingly, this embodiment illustrates a situation where graphics application 22z is unaware that there are multiple graphics pipes 50.

[0048] In operation, graphics applications 22 other than graphics application 22z, generate appropriate connection request calls, window request calls, and graphics data calls to the associated interface libraries 24, which generates appropriate requests for graphics server 30. Also, unaware application manager 32 and aware application manager 36 convey the graphics data from these clients to graphics pipes 50 for display on monitors 60. When graphics application 22z desires to have graphics data displayed, it also generates a connection request and sends it to graphics server 30 through interface library 24z. Graphics server 30 assigns the connection request to unaware application manager 32, which responds with a connection reply that aggregates the characteristics of graphics pipes 50. Then, when graphics application 22z is ready to open a window, the window request is made to graphics server 30 through interface library 24z. Unaware application manager 32 receives this window request and generates a window reply to graphics application 22z. Additionally, unaware application manager 32 notifies display system proxy 26z how graphics data is to be conveyed to graphics pipes 50. In particular embodiments, display system proxy 26z communicates with unaware application manager 32 to get the real visual identifiers and attributes of the real sibling windows the application manager created so that a matching rendering context in each window may be created. When graphics application 22z begins to send graphics data for the open window, the graphics data is intercepted by display system proxy 26z, which divides it between the appropriate ones of graphics pipes 50, and is then sent to graphics library 28z, which converts it into an appropriate format for these graphics pipes.

[0049] Note, that graphics pipes 50 are shown as bi-directional because readbacks, such as for example state commands, may be passed back from graphics pipes 50. Proxy 28z may composite all the readbacks into one for the graphics application.

[0050] This embodiment of the invention may be particularly useful when graphics application 22z is generating complex graphics data, such as for example 3D graphics, because the graphics data is sent directly to graphics pipes 50 without encountering the overhead of graphics server 30, allowing for faster rendering. If 3D graphics are being used, graphics library 26z may be an OpenGL library, which is operable to handle both OpenGL and GLX calls, or any other appropriate type of 3D graphics library.

[0051] In other embodiments, a display system proxy such as display system proxy 26z and a graphics library such as graphics library 28z may be used with one or more of graphics applications 22. Accordingly, some or all of the graphics data may avoid being sent through graphics server 30. Furthermore, a display system proxy may not be used if the graphics application is aware that there are multiple graphics pipes 50. In certain embodiments, display system proxy 26z may be disabled by an environment variable set at runtime. Note that window manager 80 and/or window manager proxy 82 of system 12 may be used to manage windows in this embodiment.

[0052] FIG. 4 is a flow diagram 400 illustrating one method for managing graphics applications. Flow diagram 400 begins at wait state 410, where a graphics server, such as for example graphics server 30, waits to receive graphics information from a graphics application. Upon receiving a connection request from a graphics application, the graphics server assigns the connection request to an application manager at function block 422 and determines the graphics resources available at function block 424. The graphics resources available may be determined, for example, by determining the capabilities of graphics pipes 50. At function block 426, the graphics server generates a connection reply for the graphics application. As mentioned previously, the reply may aggregate the characteristics of the graphics pipes or report the characteristics of the graphics pipes individually depending on whether the requesting client is aware or unaware. Once the requesting graphics application receives the connection reply, it may proceed to make other requests through the new connection. The method calls for waiting for more graphics information.

[0053] When the graphics server receives a window creation request, it associates the request with the previously assigned application manager at function block 432. The graphics server generates a window for the window request at function block 434 and generates a reply for the window request at function block 436. Once the requesting graphics application has received the window reply, the graphics application may begin to send graphics data for display. The method calls for waiting for more graphics information.

[0054] When the graphics server receives graphics data from a graphics application, the graphics server associates the data with an application manager at function block 442 based on the connection upon which the data arrives. At function block 444, the graphics data is conveyed to the appropriate graphics pipes. As discussed previously, the graphics data may be conveyed based on whether the graphics application that sent the graphics data is aware of the multiple graphics pipes or not. Once the data has been conveyed to the appropriate graphics pipes, the method calls for waiting to receive more graphics information.

[0055] As can be seen by flow diagram 400, the method allows for a variety of graphics applications to be sending graphics data to be displayed at one time. Accordingly, graphics applications that are aware of the multiple graphics pipes and those that are not so aware may be in use concurrently with their data being displayed on multiple monitors.

[0056] Other embodiments may include fewer, more, and/or a different arrangement of operations. For example, in some embodiments, the graphics server may communicate with a window manager that manages the windows. As another example, in certain embodiments, some or all of the graphics data may circumvent the graphics server. As a further example, in certain embodiments, the graphics server may associate a request with an application manager based on the type of resource being manipulated rather than the connection upon which the request arrives.

[0057] FIG. 5 illustrates an embodiment of a system 16 for managing graphics applications. As with system 10, system 16 includes clients 20, graphics server 30, and display system 40. In this embodiment, however, unaware application manager 32 includes a homogenizer 34, which, in general, is responsible for generating a common capability set for graphics pipes 50. A capability set contains attributes that specify, at least in part, how graphics data will be displayed. By using homogenizer 34, graphics applications 22 that are not aware that there are multiple graphics pipes 50 may properly create graphics data for display on monitors 60 that have different display capabilities, which expands the usefulness of multiple pipe display systems.

[0058] In operation, when one of graphics applications 22 wants to have graphics data displayed, it establishes a connection to graphics server 30. To accomplish this, the graphics application calls the associated interface library 24, which generates the appropriate graphics request for graphics server 30. When the connection request arrives at graphics server 30, graphics server 30 determines whether the connection request is for an aware graphics application or an unaware graphics application.

[0059] If for an aware graphics application, the graphics server assigns the connection request to aware application manager 36. Application manager 36 determines capability sets for each of graphics pipes 50 and sends these to the requesting graphics application with the connection reply. To determine the capability sets, application manager 36 may, for example, extract capability sets from graphics pipes 50. Application manager 36 may accomplish this, for example, with appropriate function calls and/or queries to graphics pipes 50. Upon receiving the capability sets, the requesting graphics application 22 may determine how it is going to format graphics data for each of graphics pipes 50. The graphics application may inform graphics server 30 what format it has chosen in the window request.

[0060] If, however, the connection request is for an unaware graphics application, graphics server 30 assigns the connection request to unaware application manager 32. Like application manager 36, application manager 32 may determine capability sets for each of graphics pipes 50. However, because the requesting application cannot interact with the graphics pipes individually, or does not want to interact with the graphics pipes individually, homogenizer 34 generates capability sets that are common to each of graphics pipes 50 and sends these to the requesting client with the connection reply. The graphics application of the requesting client may determine which format it will use. Note that a request for the use of only one graphics pipe may be treated similarly to the request from an aware graphics application, except that the capability sets for only one graphics pipe would be in the connection reply.

[0061] Homogenizer 34 may be used at the initiation of graphics server 30, at the initiation of one of graphics applications 22, at the initiation of each of graphics applications 22, at the initiation of each context of graphics applications 22, or at any other appropriate time. Note that the common capability sets generated by homogenizer 34 may vary depending on the capability sets already being used on graphics pipes 50.

[0062] FIG. 6 illustrates a table 600 of capability sets for graphics pipes 50a–c. As illustrated, table 600 has three columns 610. Column 610a contains the capability sets for graphics pipe 50a, column 610b contains the capability sets for graphics pipe 50b, and column 610c contains the capability sets for graphics pipe 50c. In this illustration, the first attribute in each capability set is the color scheme used, the second attribute is the number of bits used to represent color, and the third attribute is the number of buffers used for rendering graphics objects. Accordingly, for the first capability set in column 610a, “RGB” denotes a red-green-blue color scheme, “8” denotes an eight bit color representation, and “DB” denotes that two buffers are used for rendering graphics to the monitor. Additionally, some capability sets have the ability to perform stereo—denoted by “S”— and some capability sets have the ability to use auxiliary buffers—denoted by “AB”.

[0063] As can be seen in table 600, graphics pipes may have a wide range of attributes and combinations thereof. Table 600, however, only illustrates one embodiment of capability sets, and any other appropriate capability sets may be used with the present invention.

[0064] In particular embodiments, homogenizer 34 generates the common capability sets by determining which capability sets of one of graphics pipes 50 have corresponding capability sets on each of the other graphics pipes 50. Capability sets may correspond if at least one of their attributes are similar. Additionally, capability sets may not correspond if one does not contain a mandatory attribute, which may have been specified by the requesting client 20, graphics server 30, and/or a component of display system 40.

[0065] In certain embodiments, a weighting may be assigned to the attributes so that certain ones are more highly favored in determining if capability sets correspond. The relative weighting of the attributes may be specified by the requesting client 20, graphics server 30, and/or display system 40. Additionally, in particular embodiments, at least one attribute, such as for example the color scheme and/or the number of bits used to represent color, may have to match for two capability sets to correspond. These mandatory matches may also be specified by the requesting client 20, graphics server 30, and/or display system 40.

[0066] As an example, suppose that a graphics application was particularly interested in displaying graphics in stereo. Analyzing the capability sets in table 600 would lead to finding a correspondence between the first capability set in column 610a, the first capability set in column 610b, and the first capability set in column 610c, assuming color scheme and color bits received some consideration. On the other hand, if the graphics application was particularly interested in displaying graphics using two rendering buffers, a correspondence would be found, again assuming some consideration of color scheme and color bits, between: 1) the first capability set in column 610a, the second capability set in column 610b, and the first capability set in column 610c; and 2) the second capability set in column 610a, the third capability set in column 610b, and the second capability set in column 610c. As a further example, suppose that a graphics application was particularly interested in having its colors represented by twelve bits. A correspondence would be found between the third capability set in column 610a, the fourth capability set in column 610b, and the third capability set in column 610c.

[0067] Note that it may be possible for a capability set on one graphics pipe to have more than one corresponding capability set on other graphics pipes. Accordingly, the attribute weighting scheme may be used to determine the capability sets that have the highest degree of correspondence. In certain embodiments, however, capability sets that have more than one corresponding capability set on other graphics pipes may be allowed to have more than one corresponding capability set on each pipe. This may provide more options in determining the common capability sets.

[0068] After determining which capability sets have corresponding capability sets on each of graphics pipes 50, homogenizer 34 determines whether attributes that do not match between the corresponding capability sets have common characteristics. If the attributes that do not have a match between corresponding capability sets do have common characteristics, homogenizer 34 may use the common characteristics in the common capability set instead of the respective attributes.

[0069] For example, consider the first example discussed above. Determining correspondence with a heavy weight toward stereo capabilities produced three corresponding capability sets that used the RGB color scheme, eight bits to represent color, and stereo capabilities. However, the corresponding capability set in column 610a has double rendering buffers and auxiliary buffers, the corresponding capability set in column 610b has a single rendering buffer, and the corresponding capability set in column 610c has double rendering buffers and auxiliary buffers. Accordingly, there is no match between the rendering buffers and the auxiliary buffers of these capability sets. It should, however, be possible to use a single rendering buffer instead of double rendering buffers. Thus, the common capability set sent to the requesting client for these corresponding capability sets may specify that a single rendering buffer is available. Note, however, that there is no common characteristic between the auxiliary buffering attributes of these capability sets. Thus, there is no substitute for this attribute.

[0070] In general, there are a variety of attributes that have common characteristics. For example, RGB is a characteristic of RGBA and, accordingly, may be substituted for it. As another example, using eight bits to represent color is a characteristic of using sixteen bits and, accordingly, may be substituted for it. Of course, a substitution for an attribute may not be made if a component requires that attribute.

[0071] After determining whether any attributes that do not match between corresponding sets have common characteristics, homogenizer 34 determines whether the other attributes that do not match between the corresponding capability sets may not be used. This may rely upon preferences from the requesting client 20, graphics server 30, and/or the capabilities of graphics pipes 50. If these non-matching attributes may be removed, a common capability set should be achievable.

[0072] For instance, continuing with the previous example, after the double rendering buffering has been reduced to single rendering buffering, the first capability set in column 610a and the first capability set in column 610c still have auxiliary buffering capability while the first capability set in column 610b does not. If no component has specified that auxiliary buffers are mandatory, the auxiliary buffer attribute may not be used in the common capability set. Accordingly, the capability set sent to the requesting graphics application could specify an RGB color scheme using eight bits with a single rendering buffer.

[0073] As another illustration, for the case where double rendering buffers are a preferred option, the common capability set generated from the first capability set of column 610a, the second capability set of column 610b, and the first capability set of column 610c may not include the stereo capability of the first capability set of column 610a and the first capability set of column 610c to accommodate the second capability set of column 610b, as long as stereo was not a mandatory option. Furthermore, the common capability set generated from the second capability set of column 610a, the third capability set of column 610b, and the third capability set of column 610c may not include the stereo capability of the second capability set of column 610a, the auxiliary buffer capability of the third capability set of column 610b, and the stereo capability and the auxiliary buffer capability of the third capability set of column 610c, as long as stereo capability and auxiliary buffer capability were not required by a component.

[0074] After this, the common capability sets may be sent to the requesting application. If more than one common capability set is sent to the requesting application, as mentioned previously, the requesting application may choose which capability set to use.

[0075] Although system 16 illustrates one embodiment of the present invention, other embodiments may include less, more, and/or a different arrangement of components. For example, although illustrated as part of unaware application manager 32 in system 16, homogenizer 34 could be located anywhere in graphics server 30. Moreover, graphics server 30 may be configured differently too, such as for example having unaware application manager 32 be a part of aware application manager 36. Additionally, in certain embodiments, homogenizer 34 could be separate from graphics server 30. Note that homogenizer 34 may also be used with system 12 or system 14. Furthermore, homogenizer 34 may be part of a graphics server that does not manage the conveyance of graphics data from aware graphics applications, a graphics server that does manage the conveyance of graphics data from one graphics application, or otherwise associated with logic that manages the conveyance of graphics data. A variety of other examples exist.

[0076] A variety of attribute weightings may be used in determining if capability sets correspond. In certain embodiments, matching the number of bits used to represent color receives the highest priority, followed by matching the scheme used to represent color, the use of double rendering buffers, stereo capability, and auxiliary buffers, respectively. Other embodiments, of course, may use less, more, and/or a different arrangement of weightings. For example, in some embodiments, matching the color scheme may be weighted highest, especially where color schemes besides RGB and RGBA are used. As another example, in some embodiments matching the double buffering capability and stereo capabilities may not be used. As a further example, in embodiments where other attributes are present, they may receive weightings in order that they may be used in determining correspondence.

[0077] In particular embodiments, determining the correspondence between capability sets occurs in iterations. For example, during the first iteration, an algorithm performs a rough matching based on the following attributes: GLX_—RED_—SIZE, GLX_—GREEN_—SIZE, GLX_—BLUE_—SIZE, GLX_—ALPHA_—SIZE. Moreover, any other attributes based on number of bits or buffer size may be considered at this time. In the next iteration, based on a pre-specified weighting, these lists may be further culled and put together based on a lowest common denominator approach.

[0078] For example, if graphics pipe 0 has the following capability sets:

• • [0079] 1. RGBA8, DoubleBuffer, Stereo, and
  • [0080] 2. RGB12, SingleBuffer, No Stereo,
    and graphics pipe 1 has the following capability sets:
  • [0081] 1. RGB8, DoubleBuffer No Stereo, and
  • [0082] 2. RGB8, SingleBuffer, Stereo,
    then after the iteration for color bits, graphics pipe 0 will have:
  • [0083] 1. RGBA8, DoubleBuffer, Stereo,
    and graphics pipe 1 will have:
  • [0084] 1. RGB8, DoubleBuffer, No Stereo, and
  • [0085] 2. RGB8, SingleBuffer, Stereo.
    Then, if the configurable weighting specifies that double buffering has a higher weighting than stereo, after the next iteration, the final homogeneous capability set exported is:
  • [0086] 1. RGB8, DoubleBuffer, No Stereo.
    However, if the configurable weighting specifies that stereo has a higher weighting, after the next iteration, the final homogenous capability set exported is:
  • [0087] 1. RGB8, SingleBuffer, Stereo.

[0088] Table 1 presents another example of attributes and their weighting in accordance with the present invention.

[0089]

TABLE 1
Attribute Weighting
Attribute	Description
GLX—RGBA	If present, only TrueColor and
	DirectColor visuals are considered.
	Otherwise, only PseudoColor and
	StaticColor visuals are considered.
GLX—RED—SIZE	Must be followed by a nonnegative minimum
	size specification. If this value is
	zero, the smallest available red buffer
	is preferred. Otherwise, the largest
	available red buffer of at least the
	minimum size is preferred.
GLX—GREEN—	Must be followed by a nonnegative minimum
SIZE	size specification. If this value is
	zero, the smallest available green
	buffer is preferred. Otherwise, the
	largest available green buffer of at
	least the minimum size is preferred.
GLX—BLUE—	Must be followed by a nonnegative minimum
SIZE	size specification. If this value is
	zero, the smallest available blue buffer
	is preferred. Otherwise, the largest
	available blue buffer of at least the
	minimum size is preferred.
GLX—ALPHA—	Must be followed by a nonnegative minimum
SIZE	size specification. If this value is
	zero, the smallest available alpha buffer
	is preferred. Otherwise, the largest
	available alpha buffer of at least the
	minimum size is preferred.
GLX—	If present, only double-buffered visuals
DOUBLEBUFFER	are considered. Otherwise, only single-
	buffered visuals are considered.
GLX—STEREO	If present, only stereo visuals are
	considered. Otherwise, only monoscopic
	visuals are considered.
GLX—AUX—	Must be followed by a nonnegative integer
BUFFERS	that indicates the desired number of
	auxiliary buffers. Visuals with the
	smallest number of auxiliary buffers that
	meets or exceeds the specified number are
	preferred.
GLX—DEPTH—	Must be followed by a nonnegative minimum
SIZE	size specification. If this value is
	zero, visuals with no depth buffer are
	preferred. Otherwise, the largest
	available depth buffer of at least the
	minimum size is preferred.
GLX—STENCIL—	Must be followed by a nonnegative integer
SIZE	that indicates the desired number of
	stencil bitplanes. The smallest stencil
	buffer of at least the specified size is
	preferred. If the desired value is zero,
	visuals with no stencil buffer are
	preferred.
GLX—ACCUM—	Must be followed by a nonnegative minimum
RED—SIZE	size specification. If this value is
	zero, visuals with no red accumulation
	buffer are preferred. Otherwise, the
	largest possible red accumulation buffer
	of at least the minimum size is
	preferred.
GLX—ACCUM—	Must be followed by a nonnegative minimum
GREEN—SIZE	size specification. If this value is
	zero, visuals with no green accumulation
	buffer are preferred. Otherwise, the
	largest possible green accumulation
	buffer of at least the minimum size is
	preferred.
GLX—ACCUM—	Must be followed by a nonnegative minimum
BLUE—SIZE	size specification. If this value is
	zero, visuals with no blue accumulation
	buffer are preferred. Otherwise, the
	largest possible blue accumulation buffer
	of at least the minimum size is
	preferred.
GLX—ACCUM—	Must be followed by a nonnegative minimum
ALPHA—SIZE	size specification. If this value is
	zero, visuals with no alpha accumulation
	buffer are preferred. Otherwise, the
	largest possible alpha accumulation
	buffer of at least the minimum size is
	preferred.
GLX—SAMPLE—	Must be followed by the minimum
BUFFERS—SGIS	acceptable number of multisample buffers.
	Visuals with the smallest number of
	multisample buffers that meet or exceed
	this minimum number are preferred.
	Currently operation with more than one
	multisample buffer is undefined, so the
	returned value will be either zero or
	one. Note that multisampling is support
	only on RealityEngine.
GLX—SAMPLES—	Must be followed by the minimum number of
SGIS	samples required in multisample buffers.
	Visuals with the smallest number of
	samples that meet or exceed the specified
	minimum number are preferred. Note that
	it is possible for color samples in the
	multisample buffer to have fewer bits
	than colors in the main color buffers.
	However, multisampled colors maintain at
	least as much color resolution in
	aggregate as the main color buffers.
GLX—	Must be followed by one of GLX—NONE—EXT,
TRANSPARENT—	GLX—TRANSPARENT—RGB—EXT, GLX—
TYPE—EXT	TRANSPARENT—INDEX—EXT. If GLX—
	NONE—EXT is specified, then only opaque visuals
	will be considered; if GLX—TRANSPARENT—
	RGB—EXT is specified, then only transparent,
	RGBA visuals will be considered; if GLX—
	TRANSPARENT—INDEX—EXT is specified, then
	only transparent, indexed visuals will be
	considered. If not specified, the value
	GLX—NONE—EXT is used.
GLX—	Must be followed by an integer value
TRANSPARENT—	indicating the transparent index value;
INDEX—VALUE—	the value must be between 0 and the
EXT	maximum framebuffer value for indices.
	Only visuals that use the specified
	transparent index value will be
	considered. This attribute is ignored
	during visual selection unless it is
	explicitly specified and GLX—TRANSPARENT—
	TYPE—EXT is specified as GLX—
	TRANSPARENT—INDEX—EXT.
GLX—	Must be followed by an integer value
TRANSPARENT—	indicating the transparent red value; the
RED—VALUE—	value must be between 0 and the maximum
EXT	framebuffer value for red. Only visuals
	that use the specified transparent red
	value will be considered. This attribute
	is ignored during visual selection unless
	it is explicitly specified and GLX—
	TRANSPARENT—TYPE—EXT is specified as
	GLX—TRANSPARENT—RGB—EXT.
GLX—	Must be followed by an integer value
TRANSPARENT—	indicating the transparent green value;
GREEN—VALUE—	the value must be between 0 and the
EXT	maximum framebuffer value for green.
	Only visuals that use the specified
	transparent green value will be
	considered. This attribute is ignored
	during visual selection unless it is
	explicitly specified and GLX—TRANSPARENT—
	TYPE—EXT is specified as GLX—
	TRANSPARENT—RGB—EXT.
GLX—	Must be followed by an integer value
TRANSPARENT—	indicating the transparent blue value;
BLUE—VALUE—	the value must be between 0 and the
EXT	maximum framebuffer value for blue. Only
	visuals that use the specified
	transparent blue value will be
	considered. This attribute is ignored
	during visual selection unless it is
	explicitly specified and GLX—TRANSPARENT—
	TYPE—EXT is specified as
	GLX—TRANSPARENT—RGB—EXT.
GLX—	Must be followed by an integer value.
TRANSPARENT—	This attribute is always ignored; it is
ALPHA—VALUE—	for future use only.
EXT
GLX—VISUAL—	Must be followed by one of GLX—NONE—EXT,
CAVEAT—EXT	GLX—SLOW—VISUAL—EXT, GLX—NON—
	CONFORMANT—EXT. If GLX—NONE—EXT is
	specified, then only visuals with no caveats will be
	considered; if GLX—SLOW—VISUAL—EXT, is
	specified then only slow visuals will be
	considered; if GLX—NON—CONFORMANT—EXT
	is specified then only non-conformant
	visuals will be considered. This
	attribute is ignored during visual
	selection unless it is explicitly
	specified.
GLX—X—	Must be followed by one of GLX—TRUE—
VISUAL—TYPE—	COLOR—EXT, GLX—DIRECT—COLOR—EXT,
EXT	GLX—PSEUDO—COLOR—EXT, GLX—STATIC—
	COLOR—EXT, GLX—GRAY—SCALE—EXT,
	GLX—STATIC—GRAY—EXT,
	indicating the desired X visual type. If
	GLX—RGBA is in attribList, then only
	GLX—TRUE—COLOR—EXT and GLX—
	DIRECT—COLOR—EXT can
	produce a match. If GLX—X—
	VISUAL—TYPE—EXT is not in attribList, and
	if all other attributes are equivalent,
	then a TrueColor visual will be chosen in
	preference to a DirectColor visual. If
	GLX—RGBA is not in attribList, then only
	GLX—PSEUDO—COLOR—EXT and GLX—
	STATIC—COLOR—EXT can produce a match.
	If GLX—X—VISUAL—TYPE—EXT
	is not in attribList, and if all other attributes are
	equivalent, then a PseudoColor visual
	will be chosen in preference to a
	StaticColor visual.

[0090] FIG. 7 is a flow diagram 700 illustrating one embodiment of a method for managing graphics applications. The method begins at decision block 704 with waiting to receive a connection request. At decision block 708, the method calls for determining whether the connection request is for an unaware client.

[0091] If the connection request is not for an unaware client, the method calls for determining capability sets for graphics pipes at function block 712. As discussed previously, this may include extracting capability sets from graphics pipes such as graphics pipes 50. At function block 716, the method calls for generating a connection reply containing the capability sets for the graphics pipes. The method is then at an end. The requesting client will be responsible for determining which capability sets it wants to use. The requesting graphics application may communicate this selection in a window request.

[0092] If, however, the connection request is for an unaware client, the method calls for determining capability sets for graphics pipes at function block 720. At function block 724, the method calls for determining which capability sets have corresponding capability sets on the other graphics pipes. As discussed previously, this may involve analyzing which attributes are required and/or which ones are preferred. For the corresponding capability sets, the method calls for deciding if any attributes that do not have a match between them have a common characteristic at function block 728. At function block 732, if any of the attributes that do not have a match between the corresponding capability sets have a common characteristic, the method provides for using the characteristic in the common capability set instead of the attributes. Note that using a characteristic in a common capability set instead of attributes may include substituting the characteristic for the attributes in the common capability set and/or in the corresponding capability sets, inserting the characteristic in the common capability set instead of the attributes, or otherwise using the characteristic in place of the attributes in the common capability set.

[0093] At function block 736, the method calls for deciding if any attributes that do not match between corresponding capability sets and that do not have a common characteristic may not be used in the common capability set. As mentioned previously, this may be accomplished if no component has designated one of these attributes as mandatory. At function block 740, attributes that may not be used are not used in the common capability set. Note that not using the attributes in the common capability set may encompass deleting them from the common capability set and/or the corresponding capability sets, not inserting them in the common capability set, or otherwise not using them in the common capability set.

[0094] At function block 744, the method calls for determining if there is at least one common capability set. A common capability set may not exist, for example, if there is no correspondence between capability sets on each graphics pipe 50, if corresponding capability sets have attributes that do not match and that may not be deleted, or for any other appropriate reason. If there is at least one common capability set, the method calls for generating a connection reply containing the common capability sets at function block 748. The method is then at an end. As mentioned previously, if there is more than one common capability set, the requesting graphics application may determine which common capability set it desires to use. If, however, there is no common capability set, a connection reply is generated indicating that graphics service is not available at function block 752. The method is then at an end. The requesting graphics application may determine whether it wants to alter any requested attributes and submit a new connection request.

[0095] The method illustrated by flow diagram 700 may be useful with any of the previously described embodiments of the present invention. For example, the method could readily be incorporated into function block 424 of flow diagram 400.

[0096] While flow diagram 700 illustrates one embodiment of a method for managing graphics applications, other embodiments may contain less, more, and/or a different arrangement of operations. For example, in some embodiments, the method may call for removing attributes that do not have a match between corresponding capability sets and that may be removed before attempting to determine if attributes that do not match have a common characteristic. As another example, in particular embodiments, the capability sets for the graphics pipes may be maintained once they are determined the first time and, thus, when subsequent connection requests are received, the capability sets do not have to be determined again. Moreover, the capability sets may be determined before a connection request is received. As a further example, in certain embodiments, instructions may be generated for the graphics pipes as to the capability set to be used. As an additional example, in some embodiments, the failure to find a common capability set may be a fatal error and/or cause the server to reset. As a further example, in certain embodiments, if more than one common capability set exists, a downselecting between the common capability sets may be performed for the requesting client. This may be accomplished, for example, by determining which common capability set is closest to the actual capability sets. As an additional example, in particular embodiments, the operations regarding aware graphics applications may not exist, especially if there are no aware graphics applications being used. A variety of other examples exist.

[0097] FIG. 8 illustrates an embodiment of a system 910 for managing graphics applications. System 910 includes clients 920 and a graphics server 930, which are similar to clients 20 and graphics server 30 of system 10, system 12, system 14, and system 16. System 910 also includes a display system 940, which includes graphics pipes 950, projectors 960, and screens 962 to display graphics data from graphics applications 922 of clients 920. Screens 962 are arranged to provide a non-planar display. Accordingly, system 910 is particularly useful in providing an immersive atmosphere since it may occupy a large portion of the field-of-view (FOV) for viewers.

[0098] The fact that screens 962 of system 910 are non-planar produces particular difficulties in displaying graphics data from unaware graphics applications because such applications may produce graphics data assuming that screens 962 are one large planar screen, represented by 962a′–962c′. Thus, when the graphics data is displayed on screens 962, it may have a different perspective frustum and viewing direction than what the graphics application anticipated. For example, perspective frustum 970a, associated with screen 962a, is quite different from the perspective frustum for the anticipated screen 962a′. Accordingly, to improve the display of graphics data from unaware graphics applications, the non-planar viewing factors should be accommodated, at least in part. This is the responsibility of view modifier 935.

[0099] In operation, as discussed previously, graphics applications 922 call the appropriate functions in interface libraries 924 to generate connection requests, window requests, and graphics data requests. The interface libraries then send the requests to graphics server 930, where they are associated with and processed by the appropriate one of application managers 932, 936. In this embodiment, however, when an unaware graphics application wants to have graphics data displayed on one or more of screens 962, view modifier 935 may modify the graphics data so that it displays properly, at least in part, on screens 962. Modifying the graphics data may include accounting for a different perspective frustum, accounting for a different viewing angle, and/or any other appropriate visual factor. The graphics data may then be conveyed to the appropriate graphics pipes 950 for display on screens 962 by projectors 960.

[0100] View modifier 935 may accomplish its task by intercepting matrix calls and substituting the proper viewing matrices for them on a per graphics pipe basis. For example, if operating in an OpenGL environment, view modifier 935 may intercept calls for a PROJECTION matrix, which, in general, represents the FOV of the image, and a MODELVIEW matrix, which, in general, represents the viewing direction of the image. Note that the matrices anticipated by the graphics application may need to be maintained to respond to the graphics application.

[0101] In more detail, the PROJECTION matrix generally represents a perspective frustum, although it can be used to represent a variety of other views, such as, for example, orthogonal. In operation, calls to create the frustum:

• • [0102] glFrustum(l, r, b, t, n, f),
    where
  • [0103] l relates to the left side of the frustum,
  • [0104] r relates to the right side of the frustum,
  • [0105] b relates to the bottom of the frustum,
  • [0106] t relates to the top of the frustum,
  • [0107] n relates to the near field of the frustum, and
  • [0108] f relates to the far field of the frustum, could be replaced by:
  • [0109] glFrustum(l_p, r_p, b_p, t_p, n_p, f_p)
    where
  • [0110] p represents the graphics pipe under consideration. The current matrix may then be multiplied by the perspective matrix to produce the appropriate perspective projection.

[0111] The variables that define the new frustum may be computed, at least in part, by specifying the configuration of screens 962 associated with each graphics pipe 950. In particular embodiments, the configuration of screens 962 may be specified by the bottom right corner, the bottom left corner, and the top left corner of the screens relative to the viewer being located at an origin located near the center of the surface partially described by screens 962. In other embodiments, the configuration of screens 962 may be specified by the projection origin, the horizontal FOV, the vertical FOV, the horizontal and vertical projection offsets plus roll, and the distance from the origin to the screen. In general, any appropriate type of characteristics may be used to specify the configuration of screens 962. Furthermore, the screen configuration characteristics may be specified in a configuration file that is parsed at run-time, by default values for view modifier 935, by characteristics received from a user during execution, and/or otherwise received by view modifier 935.

[0112] Additionally, calls to glLoadMatrix(M), which is typically used to replace the current matrix—projection matrix, modelview matrix, or texture matrix, depending on the current matrix mode—with the specified matrix, could be replaced by glFrustum(l_p, r_p, b_p, t_p, n_p, f_p) if M corresponds to a frustum(l, r, b, t, n, f), unless M represents an orthogonal projection. Furthermore, calls to glMultMatrix(M), which multiplies the current matrix with the specified matrix, could be replaced by glFrustum(l_p, r_p, b_p, t_p, n_p, f_p) if the resulting matrix M′ will correspond to a frustum(l, r, b, t, n, f), unless M′ represents an orthogonal projection.

[0113] In regards to the MODELVIEW matrix, the aiming direction is modified by applying a rotation centered at the projection system origin. This means that the MODELVIEW matrix should be initialized by a modified rotation matrix, R_p, where p corresponds to the graphics pipe under consideration, instead of the typical identity matrix. In operation, calls to glLoadIdentity( ), which replaces the current matrix with the identity matrix (semantically equivalent to calling glLoadMatrix with the identity matrix, but in some cases more efficient), could be replaced with calls to glLoadMatrix(R_p). Additionally, calls to glLoadMatrix(M) may be replaced by calls to glLoadMatrix(R_p) and glMultMatrix(M).

[0114] View modifier 935 may be used at the initiation of graphics server 930, at the initiation of one of graphics applications 922, at the initiation of each of graphics applications 922, at the initiation of each context of graphics applications 922, and/or at any other appropriate time. Moreover, view modifier 935 may be used while a graphics application is operating to update viewing characteristics.

[0115] As an example of the latter, some immersion approaches may be window geometry dependent. Accordingly, the frustum and rotation may need to be updated dynamically as the window for the graphics data is moved and/or resized. One way of handling the updating of the matrices in the OpenGL environment is to use a matrix stack. Thus, using push/pop mechanisms, such as, for example, glPopMatrix( ) or glPushMatrix( ), the matrix may be re-initialized to a given value. The matrix stack may be of any appropriate size.

[0116] In using the matrix stack, there is a current level. In this context, R_p is applied at a given stack level, R_p—Level. Every time R_p changes, R_p—Level is set to the maximum MODELVIEW stack level. Thus, if R_p—Level is greater than the current matrix level, M is set to the current graphics application specified matrix, the functions glLoadMatrix(R_p) and glMultMatrix(M) are called, and R_p—Level is set to the current matrix level.

[0117] Although discussed in the context of OpenGL, this embodiment of the invention may, in general, be used with any 3D graphics application program interface that has matrix calls, such as, for example, OpenGL or Direct 3D. Furthermore, although illustrated as part of unaware application manager 932 in system 910, view modifier 935 could be located anywhere in graphics server 930. Moreover, graphics server 930 may be configured differently too, such as, for example, having unaware application manager 932 be a part of aware application manager 936. Additionally, in certain embodiments, view modifier 935 could be separate from graphics server 930.

[0118] In general, view modifier 935 may be used with any display system that has a non-planar display surface. Moreover, view modifier 935 may also be used to perform any other rendering features that involve altering graphics data from the geometry of the display system expected by the graphics application for the actual geometry of the display system, such as for example curved screen to flat screen and/or non-planar screen to non-planar screen, which could be useful for aware and/or unaware graphics applications.

[0119] Display system 940 of system 910 may also have a variety of configurations. For example, screens 962 may be flat, curved, and/or any other appropriate shape. In some embodiments where the screens are curved, projectors 960 may optically compensate for the curvature of the screens. As another example, projectors 960 may be digital light projectors, liquid crystal display projectors, and/or any other type of device for generating visual information. Note that any number of projectors and screens may be used. In other embodiments, however, projectors 960 and screens 962 may be replaced by monitors and/or other appropriate devices for producing visually perceptible images.

[0120] As mentioned previously, there are differing immersion approaches that may be used. Thus, there are different ways to compute the values that will be used to modify the graphics data from a graphics application. These values may be dictated by the geometry of the display configuration, the expectations of the graphics application, and/or any other appropriate factor.

[0121] One immersion approach is to base the modifications on the geometry of screens 962, a display driven approach. This approach has the advantage that it is perceptually truly immersive to a viewer. In operation, the frustum definitions anticipated by the graphics application are replaced by a frustum defined by the display layout.

[0122] In particular embodiments, this may be accomplished by replacing the l, r, b, and t values of the frustum anticipated by the graphics application by values computed from the screen layout, the f and n values of the frustum for the graphics application being retained. Given the bottom left, top left, and bottom right definition of a screen, l_p, r_p, b_p, and t_p for the corresponding frustum may be determined by computing the directional vectors for the screen and the orientation of the screen relative to the viewer. The directional vectors for the screen may be computed by:
Ū={overscore (win)}_{bottom—right}−{overscore (win_{bottom—left})};
{overscore (V)}={overscore (win)}_top—left−{overscore (win_{bottom—left})}; and
{overscore (W)}=Ū×{overscore (V)}.
The orientation of the screen relative to the viewer may be computed by:
eye_x=−{overscore (win_center)}•Ū;
eye_y=−{overscore (win_center)}•{overscore (V)}; and
eye_z=−{overscore (win_center)}•{overscore (W)}.
Then, l_p, r_p, b_p, and t_p for the frustum associated with the graphics pipe may be computed by: $l_p=-\left(\frac{{\mathrm{win}}_{\mathrm{width}}}{2}+{\mathrm{eye}}_x\right)*\mathrm{dratio}$ $r_p=\left(\frac{{\mathrm{win}}_{\mathrm{width}}}{2}-{\mathrm{eye}}_x\right)*\mathrm{dratio}$ $b_p=-\left(\frac{{\mathrm{win}}_{\mathrm{height}}}{2}+{\mathrm{eye}}_y\right)*\mathrm{dratio}$ $t_p=\left(\frac{{\mathrm{win}}_{\mathrm{height}}}{2}-{\mathrm{eye}}_y\right)*\mathrm{dratio}$ $\mathrm{where}$ $\mathrm{dratio}=\frac{n}{{\mathrm{eye}}_z}.$
Also as a result of these calculations, the frustum matrix may be determined: $\mathrm{Frustum}=\left[\begin{array}{cccc}{U}_x& U_y& U_z& {\mathrm{eye}}_x\\ V_x& V_y& V_z& {\mathrm{eye}}_y\\ W_x& W_y& W_z& {\mathrm{eye}}_z\\ a=0& b=0& c=0& \mathrm{dratio}\end{array}\right].$
Additionally, from the frustum matrix, the rotation matrix, R_p, may be computed: $R_p=\left[\begin{array}{cccc}{U}_x-{\mathrm{eye}}_x*a& U_y-{\mathrm{eye}}_x*b& U_z-{\mathrm{eye}}_x*c& {\mathrm{eye}}_x-{\mathrm{eye}}_x*\mathrm{dratio}\\ V_x-{\mathrm{eye}}_y*a& V_y-{\mathrm{eye}}_y*b& V_z-{\mathrm{eye}}_y*c& {\mathrm{eye}}_y-{\mathrm{eye}}_y*\mathrm{dratio}\\ W_x-{\mathrm{eye}}_z*a& W_y-{\mathrm{eye}}_z*b& W_z-{\mathrm{eye}}_z*c& {\mathrm{eye}}_z-{\mathrm{eye}}_z*\mathrm{dratio}\\ a& b& c& \mathrm{dratio}\end{array}\right].$

[0123] For graphics applications not running in full screen mode, the view modification is this immersion approach may be dependent on the geometry of the window instead of the screen. The application window can be thought of as a real window through which the scene is viewed. Thus, moving the window will result in seeing a different part of the scene. Accordingly, the projection correction is dependent on the position of the window. For example, the scene may appear differently when the window is fully contained in the front screen than when it is overlapping two screens. In this case, the bottom left, top left, and bottom right values correspond to the window's corners, expressed in the same coordinate system as the screen.

[0124] To keep the same viewing direction when moving the window, meaning that the observer would see the same part of the scene when the window is moved relative to the straight forward gaze of the eye, another rotation matrix, R_w, could be added to the view orientation matrix, such as for example the MODELVIEW matrix, resulting in R_p being replaced by R_w×R_p. This matrix would not be pipe dependent and would provide window dependant rotation around the origin.

[0125] One problem that may occur with the display driven approach is that the frustum used by the graphics application and the one determined from the display system layout may differ substantially, leaving objects with different appearances on the screen. In particular embodiments, a user may provide placement hints, perhaps in a configuration file, for the graphics data. The hints may be turned into calls to rotate, scale, translate, and/or otherwise position the graphics data. The alternate calls involving rotation could then be followed by the hint calls before proceeding with application-related calls. For example, in the OpenGL context, the glLoadMatrix(M) call would be modified to:

• • [0126] glLoadMatrix(R_p);
  • [0127] glRotate/glScale/glTranslate; and
  • [0128] glMultMatrix(M).
    In other embodiments, a semi-automated modification could be made by having a user match the appearance of the scene bounding-box drawn by the application frustum and that drawn by the display driven frustum. Of course, in certain embodiments, a computer could perform the matching automatically.

[0129] Another immersion approach is to base the modifications on the expectations of the graphics application. In this approach, the origin is moved along the z-axis in order to have the screen or window geometry match the application's frustum. In other words, translate along the z-axis the bottom left, top left, and bottom right coordinates of the graphics pipes so that the outermost edges of the display layout match the corresponding sides of the application's frustum. Once this is accomplished, the mathematics is the same as for the display driven approach. While this solution is relatively straightforward, the quality of the immersion perceived by the user may be decreased. Another issue is that since the display layout is not planar, the position of a 2D window and the 3D objects drawn within its borders may not be perfectly matched.

[0130] For some graphics packages, such as, for example, OpenGL, an assumption exists that the display is planar. Thus, graphics applications will assume that the near and far clip boundaries are planes perpendicular to the view direction (parallel to the surface of the screen). This is not always the case, however, when the display is non-planar. In the illustrated embodiment, each of screens 962 has far and near clip planes, collectively making the clip boundary curved. For example, if each pipe has its far clip plane set at the same distance from the eye point, a concave curved screen would have a concave clip boundary. Thus, when an object is moved sideways along the x-axis onto a side graphics pipe, the object could be prematurely clipped from the scene if it encounters the far clipping plane of the side pipe. To overcome this, the distances of the near and far clip planes of each pipe may be adjusted to make sure that all of the geometry is displayed.

[0131] FIG. 9 illustrates an embodiment of a system 912 for managing graphics applications. As with system 910, system 912 includes clients 920, graphics server 930, and display system 940. Moreover, projectors 960 and screens 962 are arranged to provide a non-planar display. In this embodiment, however, client 920z includes a display system proxy 926z, which includes view modifier 935, and a graphics library 928z. Display system proxy 926z is operable to intercept and divide the graphics data between graphics pipes 950, and graphics library 928z is operable to convert the graphics data from graphics application 922z into an appropriate format for display system 940. The data is sent to graphics pipes 950 using render threads, and proxy 926z may create one thread per pipe per context per application. Note, that the graphics pipes are shown as bi-directional because the state of the graphics pipes may be queried by the client and passed back from display system 940. Proxy 926z may composite all the readbacks into one for graphics application 9.22z. Accordingly, this embodiment illustrates a situation where graphics application 922z is unaware of the true number of graphics pipes.

[0132] In operation, graphics applications 922, other than graphics application 922z, call the appropriate functions in interface libraries 924 to generate connection requests and window requests for graphic server 930. Furthermore, other than graphics application 922z, graphics applications 922 call the appropriate functions in interface libraries 924 to generate graphics data requests that unaware application manager 932 and aware application manager 936 convey to display system 940 for display on screens 962. When graphics application 922z wishes to have its graphics data displayed, it, as before, calls the appropriate function in interface library 924z, which generates a connection request. Graphics server 930 assigns this connection request to unaware application manager 932, which responds with a connection reply that aggregates the characteristics of graphics pipes 950. Then, when graphics application 922z wants to open a window, it calls the appropriate function in interface library 924z, which sends a window request to graphics server 930. Unaware application manager 932 receives this window request and generates a window and reply to graphics application 922z. When graphics application 922z begins to send graphics data for the open window, the graphics data will be intercepted by display system proxy 926z, which divides it between the appropriate ones of graphics pipes 950. Display system proxy 926z may query unaware application manager 932 to determine how graphics data is to be divided between graphics pipes 950. Additionally, view modifier 935 modifies the graphics data on a per pipe basis to account for the non-planar layout of screens 962. The data will then be sent to graphics library 928z, which converts the data into an appropriate format for display system 940.

[0133] This embodiment of the invention may be particularly useful when graphics application 922z is generating complex graphics data, such as for example 3D graphics, because the graphics data is sent directly to display system 940 without encountering the overhead of graphics server 930, allowing for faster rendering. If 3D graphics are being used, graphics library 928z may be an OpenGL library or any other appropriate type of 3D graphics library.

[0134] In other embodiments, a display system proxy such as display system proxy 926z, and a graphics library such as graphics library 928z may be used with one or more of graphics applications 922. Accordingly, some or all of the graphics data may avoid being sent through graphics server 930. Furthermore, a display system proxy may not be used if the graphics application is aware of the graphics pipes. In certain embodiments, the display system proxy may be disabled by an environment variable set at runtime. Additionally, although illustrated as part of display system proxy 926z in system 912, a view modifier may be located anywhere in a client. Moreover, in certain embodiments, a view modifier may be separate from a client. The view modifier may also be used with any of system 10, system 12, system 14, system 16, or any of a variety of other embodiments of graphics server 30. Note that the window manager and/or window manager proxy of system 12 may still be used to manage windows.

[0135] FIG. 10 illustrates an embodiment of a display system 980 that may be used to display graphics data. This embodiment may be particularly useful with systems such as system 910 and system 912. As illustrated, display system 980 includes screens 990 that are curved and provide an immersive environment for a viewing location 982. In operation, each of screens 990 may be illuminated by a different projector.

[0136] For unaware applications, each of screens 990 displays graphics data that was originally meant for a planar screen, represented by 990a′–c′. For example, screen 990a displays graphics data originally intended for the part of the entire (logical) screen represented by 990a′. In particular embodiments, optics in the projectors can correct for any distortions caused by the curvature of screens 990.

[0137] In order to have the correct perception of immersion, points generated for the planar screens 990a′–990c′ may have to be modified for projection on screens 990. For example, without modification, point 992′ on screen 990a′ projects to point 992 on screen 990a. However, the object should be drawn at point 994 to provide the appropriate perception at viewing location 982. View modifier 935 may perform the appropriate corrections.

[0138] FIG. 11 is a flow diagram 1100 illustrating one embodiment of a method for managing graphics applications. The method begins a decision block 1104 with waiting to receive graphics data. Upon receiving the graphics data, the method calls for determining whether the graphics data is from an unaware graphics application or an aware graphics application at decision block 1108. As mentioned previously, this may be accomplished, for example, by examining the connection upon which the graphics data arrives. If the graphics data is from an aware graphics application, the method continues with distributing the graphics data between graphics pipes at function block 1112. If, however, the graphics data is from an unaware graphics application, the method calls for dividing the data between graphics pipes at function block 1116. At function block 1120, the method calls for modifying the graphics data based on the display for each pipe. As discussed previously, this may include modifying a perspective frustum and viewing direction for the graphics data for the graphics pipe. The graphics data is sent to the graphics pipes at function block 1124.

[0139] While flow diagram 1100 illustrates one embodiment for managing graphics applications in accordance with the present invention, other embodiments may have less, more, and/or a different arrangement of operations. For example, in certain embodiments, the modifications to the graphics data for each graphics pipe may be determined. As another example, in some embodiments, modifying the graphics data may include intercepting matrix calls and substituting the proper matrices on a per graphics pipe basis. As an additional example, in certain embodiments, the modifications may be updated if the graphics data is moved from one location to another on the display. A variety of other examples exist.

[0140] Although the invention has been described through the use of several embodiments, those skilled in the art will readily recognize a variety of additions, deletions, substitutions, and transformations that may be made to the illustrated embodiments. Accordingly, the following claims are intended to encompass such additions, deletions, substitutions, and transformations.